Logic Regression

Logic regression is an adaptive regression methodology that attempts to construct predictors as Boolean combinations of binary covariates. In many regression problems a model is developed that relates the main effects (the predictors or transformations thereof) to the response, while interactions are usually kept simple (two- to three-way interactions at most). Often, especially when all predictors are binary, the interaction between many predictors may be what causes the differences in response. This issue arises, for example, in the analysis of SNP microarray data or in some data mining problems. In the proposed methodology, given a set of binary predictors we create new predictors such as “X₁, X₂, X³, and X⁴ are true,” or “X⁵ or X⁶ but not X⁷ are true.” In more specific terms: we try to fit regression models of the form g(E[Y]) = b₀ + b₁ L₁ + · · · + b_n L_n , where L_j is any Boolean expression of the predictors. The L_j and b_j are estimated simultaneously using a simulated annealing algorithm. This article discusses how to fit logic regression models, how to carry out model selection for these models, and gives some examples.     

Far Casting Cross-Validation

Cross-validation has long been used for choosing tuning parameters and other model selection tasks. It generally performs well provided the data are independent, or nearly so. Improvements have been suggested which address ordinary cross-validation’s (OCV) shortcomings in correlated data. Whereas these techniques have merit, they can still lead to poor model selection in correlated data or are not readily generalizable to high-dimensional data.

The proposed solution, far casting cross-validation (FCCV), addresses these problems. FCCV withholds correlated neighbors in every aspect of the cross-validation procedure. The result is a technique that stresses a fitted model’s ability to extrapolate rather than interpolate. This generally leads to better model selection in correlated datasets.

Whereas FCCV is less than optimal in the independence case, our improvement of OCV applies more generally to higher dimensional error processes and to both parametric and nonparametric model selection problems. To facilitate introduction, we consider only one application, namely estimating global bandwidths for curve estimation with local linear regression. We provide theoretical motivation and report some comparative results from a simulation experiment and on a time series of annual global temperature deviations. For such data, FCCV generally has lower average squared error when disturbances are correlated.

Supplementary materials are available online.     

Singular Value Decomposition and Its Visualization

Singular value decomposition (SVD) is a useful tool in functional data analysis (FDA). Compared to principal component analysis (PCA), SVD is more fundamental, because SVD simultaneously provides the PCAs in both row and column spaces. We compare SVD and PCA from the FDA view point, and extend the usual SVD to variations by considering different centerings. A generalized scree plot is proposed to select an appropriate centering in practice. Several useful matrix views of the SVD components are introduced to explore different features in data, including SVD surface plots, image plots, curve movies, and rotation movies. These methods visualize both column and row information of a two-way matrix simultaneously, relate the matrix to relevant curves, show local variations, and highlight interactions between columns and rows. Several toy examples are designed to compare the different variations of SVD, and real data examples are used to illustrate the usefulness of the visualization methods.     

Visualization of Multivariate Density Estimates With Shape Trees

This article introduces graphical tools for visualizing multivariate functions, specializing to the case of visualizing multivariate density estimates. We visualize a density estimate by visualizing a series of its level sets. From each connected part of a level set a shape tree is formed. A shape tree is a tree whose nodes are associated with regions of the level set. With the help of a shape tree we define a transformation of a multivariate set to a univariate function. The shape trees are visualized with the shape plots and the location plot. By studying these plots one may identify the regions of the Euclidean space where the probability mass is concentrated. An application of shape trees to visualize the distribution of stock index returns is presented.     

Visualizing Online Auctions

Online auctions have been the subject of many empirical research efforts in the fields of economics and information systems. These research efforts are often based on analyzing data from Web sites such as eBay.com which provide public information about sequences of bids in closed auctions, typically in the form of tables on HTML pages. The existing literature on online auctions focuses on tools like summary statistics and more formal statistical methods such as regression models. However, there is a clear void in this growing body of literature in developing appropriate visualization tools. This is quite surprising, given that the sheer amount of data that can be found on sites such as eBay.com is overwhelming and can often not be displayed informatively using standard statistical graphics. In this article we introduce graphical methods for visualizing online auction data in ways that are informative and relevant to the types of research questions that are of interest. We start by using profile plots that reveal aspects of an auction such as bid values, bidding intensity, and bidder strategies. We then introduce the concept of statistical zooming (STAT-zoom) which can scale up to be used for visualizing large amounts of auctions. STAT-zoom adds the capability of looking at data summaries at various time scales interactively. Finally, we develop auction calendars and auction scene visualizations for viewing a set of many concurrent auctions. The different visualization methods are demonstrated using data on multiple auctions collected from eBay.com.     

Fast Nonparametric Quantile Regression With Arbitrary Smoothing Methods

The calculation of nonparametric quantile regression curve estimates is often computationally intensive, as typically an expensive nonlinear optimization problem is involved. This article proposes a fast and easy-to-implement method for computing such estimates. The main idea is to approximate the costly nonlinear optimization by a sequence of well-studied penalized least squares-type nonparametric mean regression estimation problems. The new method can be paired with different nonparametric smoothing methods and can also be applied to higher dimensional settings. Therefore, it provides a unified framework for computing different types of nonparametric quantile regression estimates, and it also greatly broadens the scope of the applicability of quantile regression methodology. This wide applicability and the practical performance of the proposed method are illustrated with smoothing spline and wavelet curve estimators, for both uni- and bivariate settings. Results from numerical experiments suggest that estimates obtained from the proposed method are superior to many competitors. This article has supplementary material online.     

A Sparse Structured Shrinkage Estimator for Nonparametric Varying-Coefficient Model With an Application in Genomics

Many problems in genomics are related to variable selection where high-dimensional genomic data are treated as covariates. Such genomic covariates often have certain structures and can be represented as vertices of an undirected graph. Biological processes also vary as functions depending upon some biological state, such as time. High-dimensional variable selection where covariates are graph-structured and underlying model is nonparametric presents an important but largely unaddressed statistical challenge. Motivated by the problem of regression-based motif discovery, we consider the problem of variable selection for high-dimensional nonparametric varying-coefficient models and introduce a sparse structured shrinkage (SSS) estimator based on basis function expansions and a novel smoothed penalty function. We present an efficient algorithm for computing the SSS estimator. Results on model selection consistency and estimation bounds are derived. Moreover, finite-sample performances are studied via simulations, and the effects of high-dimensionality and structural information of the covariates are especially highlighted. We apply our method to motif finding problem using a yeast cell-cycle gene expression dataset and word counts in genes’ promoter sequences. Our results demonstrate that the proposed method can result in better variable selection and prediction for high-dimensional regression when the underlying model is nonparametric and covariates are structured. Supplemental materials for the article are available online.     

A Penalized Likelihood Approach to Rotation of Principal Components

A new paradigm for enhancing the interpretability of principal components through rotation is presented within the framework of penalized likelihood. The rotated components are computed as the maximizers of a Gaussian-based profile log-likelihood function plus a penalty term defined by a standard rotation criterion. This method enjoys a number of advantages over other methods for principal component rotation, notably (1) the rotation specifically targets ill-defined principal components, which may benefit the most from rotation, and (2) the connection with likelihood allows assessment of the fidelity of the rotated components to the data, thereby guiding the choice of penalty parameter. The method is illustrated with an application to a small functional dataset. Efficient computation of the penalized likelihood solution is possible using recently developed algorithms for optimization under orthogonality constraints.     

Analysis and Validation of a Mathematical Model for Intracranial Pressure Dynamics

Lumped parameter, compartmental models provide a promising Method for mathematically studying the dynamics of human intracranial pressure. In this modeling approach, a system of fully time-dependent differential equations for interacting compartmental pressures is obtained by considering the intracranial system to be confined within the almost-rigid skull and developing continuity equations associated with conservation of mass. Intracranial volumes and flows are related to compartmental pressure differences through compliance and resistance parameters. In the nonlinear case where compliances are not constant, there is a lack of physical information about these parameters. Consequently, it is vital that any mathematical model with an assumed pressure-dependent compliance be validated through comparison with experimental data. The present work develops a logistic representation for the compliance between the cerebrospinal fluid and brain matter compartments. The nonlinear mathematical model involving this logistic compliance is validated here by comparing its predicted response for bolus injections of cerebrospinal fluid to laboratory data generated in an animal model. Comparison with the animal studies fully supports the validity of the mathematical model with the logistic compliance.     

Error estimate of eigenvalues of discrete linear Hamiltonian systems with small perturbation

In this paper, eigenvalues of perturbed discrete linear Hamiltonian systems are considered. A new variational formula of eigenvalues is first established. Based on it, error estimates of eigenvalues of systems with small perturbation are given under certain non-singularity conditions. Small perturbations of the coefficient functions, the weight function and the coefficients of the boundary condition are all involved. As a direct consequence, continuous dependence of eigenvalues on boundary value problems is obtained under the non-singularity conditions. In addition, two examples are presented to illustrate the necessity of the non-singularity conditions and the complexity of the problem in the singularity case.